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Summary 


Description  of  Concern 

The  ETMS  program  is  a  Naval  officer  postgraduate  degree  program  which  was  initiated  in  the  early 
1970's.  It  is  offered  through  civilian  universities,  and  results  in  a  Masters  of  Arts  degree  in 
Education  and  the  award  of  a  Navy  subspecialty  code  as  an  Education  and  Training  specialist. 
Eight  universities  currently  sponsor  this  program  with  various  curricula,  but  in  each  case 
successful  completion  results  in  a  MA  in  Education  and  subspecialty  designation.  The  Chief  of 
Naval  Education  and  Training  (CNET)  office  in  Pensacola,  Florida  directs  this  program,  creating 
education-skill  requirements  (ESR's)  and  then  matching  course  offerings  with  Navy  requirements 
to  ensure  a  certain  level  of  consistency  throughout  the  varying  degree  requirements.  The  match 
between  skill  requirements  and  fleet  needs  is  crucial  to  the  success  of  the  program.  Graduates  of 
the  ETMS  program  are  expected  to  go  on  in  their  Naval  careers  working  in  various  positions  in 
areas  such  as  education  and  training,  instructional  design,  and  evaluation.  There  are  currently  142 
graduates  now  in  the  fleet.  Periodic  evaluation  of  the  program  is  required  to  ensure  that  the 
program  continues  to  evolve  in  response  to  fleet  needs. 

Recommendations 

1 .  A  second  evaluation  should  be  conducted  targeting  only  and  all  of  the  XX37P  coded 
officers  currently  in  the  fleet. 

2 .  Naval  Military  Personnel  Command  should  be  alerted  of  the  underutilization  of  ETMS 
officers  and  make  adjustments  in  the  assignment  procedure  to  ensure  that  subspecialists  are 
given  a  utilization  tour  within  the  first  two  tours  after  graduation  as  specified  by  OPNAV. 

3 .  The  follow  up  evaluation  should  be  of  sufficient  size  that  only  the  responses  of  XX37P 
coded  officers  who  have  been  or  are  in  a  utilization  tour  are  used  to  determine  the  relevance 
of  their  education  to  fleet  ETMS  jobs. 

4 .  Arrangements  should  be  made  with  the  civilian  institutions  to  ensure  that  their  ETMS 
program  curricula  meets  all  of  the  required  objectives. 

5 .  An  optional  recommendation  is  that  an  ESR  course  be  taught  at  all  host  civilian  institutions 
(or  at  a  local  base)  by  Navy  instructors.  An  example  of  just  such  a  course  has  been  created 


by  an  XX37P  coded  officer  and  will  be  taught  this  fall  at  Old  Dominion  University.  For 
more  information  See  Appendix  E. 

Rationale 

The  concept  of  relevancy  of  education  to  job  needs  is  of  great  importance  to  both  universities  and 
employers.  The  majority  of  research  that  supports  this  statement  falls  into  two  broad  categories; 
graduate's  evaluation  of  the  match  between  curricula  and  required  job  skills,  and  employers 
evaluation  of  graduates  job  skills,  performance,  and  knowledge.  Studies  have  determined  that  the 
more  that  is  known  about  competencies  needed  in  careers  and  are  taken  into  account  in  curriculum 
development  the  more  competitive  graduates  could  be.  Educational  researchers  have  found 
evidence  that  follow-up  studies  have  great  value  to  any  college,  yet  they  have  also  found  that 
follow-up  studies  of  graduates  to  determine  their  perceptions  of  the  strengths  and  weaknesses  of 
curricula  are  infrequently  undertaken 

Current  research  demonstrates  the  ongoing  need  by  both  educational  institutions  and  employers  for 
continuing  studies  that  evaluate  the  balance  between  employers  needs  and  the  curriculum  provided 
for  students.  Such  evaluations  will  help  create  a  dynamic  and  systematic  relationship  between 
educational  institutions  and  the  work  place. 

Implications 

There  should  be  no  major  cost  effects  involved  other  than  those  normally  accrued  during  the 
biannual  on  site  curriculum  evaluation.  The  possible  exception  to  this  being  the  creation  of  a 
"Navy  ESR"  course.  As  one  ETMS  officer  has  already  created  the  course,  it  would  merely 
necessitate  the  duplication  costs  and  the  assignment  of  appropriate  civilians  or  NROTC  staff  to 
instruct  the  course  at  site  universities. 
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PROGRAM  DESCRIPTION 


Background 


The  Education  and  Training  Management  Subspecialty  (ETMS)  program  is  a  Navy  officer  postgraduate 
degree  program  which  was  initiated  in  the  early  1970's.  It  is  offered  through  various  civilian  universities, 
and  results  in  the  completion  of  a  Masters  of  Arts  degree  in  Education  and  the  award  of  a  Navy  subspecialty 
code  as  an  Education  and  Training  specialist.  Eight  universities  currently  sponsor  this  program  with 
various  curricula,  but  in  each  case  successful  completion  results  in  a  MA  in  Education  and  subspecialty 
designation.  The  Chief  of  Naval  Education  and  Training  (CNET)  office  in  Pensacola,  Florida  directs  this 
program,  matching  course  offerings  with  Navy  requirements  to  ensure  a  certain  level  of  consistency 
throughout  the  varying  degree  requirements.  Graduates  of  the  ETMS  program  are  expected  to  go  on  in 
their  Naval  careers  working  in  various  positions  in  areas  such  as  education  and  training,  instructional 
design,  and  evaluation.  There  are  currently  142  graduates  who  are  out  in  the  fleet. 


Evaluation  Purpose 


•  Are  the  knowledge  and  skills  defined  in  the  Education  and  Training  Management  Subspecialty 
(ETMS)  Educational  Skills  Requirements  (ESR),  acquired  by  program  participants? 

•  Are  the  skills  and  knowledge  acquired  with  the  ETMS  Masters  of  Arts  degree  relevant  and 
applicable  to  Navy  education  jobs  in  the  fleet? 

•  What  did  Navy  officers  like  and  dislike  about  the  ETMS  program,  and  what  would  they  do  to 
improve  it? 

•  And  a  question  ultimately  raised,  though  not  addressed  in  this  study,  do  Navy  officers  find 
certain  universities'  curricula  more  relevant  to  fleet  jobs  than  others? 

These  are  the  questions  that  this  evaluation  hopes  to  provide  answers  and  insights  to. 

The  primary  purpose  for  this  evaluation  is  to  determine  the  level  ESR  knowledge  acquired,  and  if  the  skills 
gained  from  the  ETMS  program  are  useful  and  applicable  to  actual  education  jobs  in  the  fleet  Additionally, 
it  is  hoped  that  information  regarding  the  most  optimal  curricula/courses  (or  combination  of 
curricula/courses)  for  the  ETMS  program  would  be  identified.  This  secondary  information  would  allow,  at 
a  later  time,  a  comparison  among  the  various  universities'  curricula  for  potential  change  and  improvement. 
The  evaluation  project  examined  the  opinions  of  ETMS  graduates  and  their  supervisors  regarding  their  level 
of  ESR  knowledge  acquisition,  and  the  relevancy  and  applicability  of  their  degree  programs  to  their  current 
jobs.  Questionnaires  and  interviews  were  triangulated  to  access  attitudes  and  opinions  about  the  program. 
It  is  hoped  that  the  summary  of  findings  will  be  used  to  trigger  further  evaluation  of  individual  university 


curricula  for  optimal  course  determination,  as  well  as  be  used  for  by  decision-making  groups  to  modify,  or 
improve  the  existing  programs  to  more  readily  meet  fleet  needs. 

Issues  /  Concerns 

The  primary  audience  for  this  evaluation  is  CNET.  Dr.  J.D.  Smith,  head  of  Civilian  Institutions  Programs 
at  CNET  is  the  client.  The  secondary  audience  is  Navy  Postgraduate  School  at  Monterey,  Civilian 
Institutions  coordinator,  LT  Padgett.  Stakeholders  would  be  the  eight  universities  with  ETMS  programs: 
George  Washington  University,  Harvard  University,  Memphis  State  University,  Old  Dominion  University, 
San  Diego  State  University,  Stanford  University,  University  of  West  Florida  and  recently  added  University 
of  Rhode  Island.  Current  and  future  ETMS  students  will  also  be  stakeholders. 

There  are  political  implications  concerning  the  various  civilian  university  curricula  being  scrutinized, 
however  as  routine  changes  and  upgrade  of  the  program  occur  based  on  the  Navy's  requirements,  potential 
problems  should  be  minimized.  This  evaluation  should  prove  to  be  a  valuable  tool  for  the  existing  program 
verification  system.  Both  clients,  CNET  and  Navy  Postgraduate  School,  are  very  supportive  and  interested 
in  this  project. 


METHODOLOGY 


Evaluation  Design 


A  formative  evaluation  design  was  selected  utilizing  a  goal  oriented/decision  focussed  approach  The  goal 
oriented  approach  was  identified  because  of  the  clearly  defined  objectives  of  the  ETMS  program  which 
facilitate  agreement  scale  self-report  measurement  This  approach  was  not  chosen  singularly,  however,  as 
the  program  is  ongoing  and  requires  decision-focussed  evaluation  information  for  potential  change  and  or 
improvement. 

A  qualitative  evaluation  was  indicated  due  to  the  extraneous  variables  that  could  influence  or  invalidate 
quantitative  measurements.  Any  tangible  measurement  (promotion,  job  selection,  awards  etc...)  of  an 
ETMS  officer's  competency/success  in  the  field  could  not  be  isolated  from  such  random  variables  as  time  in 
rank,  fleet  requirements,  personality  traits,  and  billet  opportunities. 


Sampling 


The  respondents  for  the  questionnaire  were  a  sample  of  ETMS  subspecialty  Naval  officers  currently  in  the 
fleet.  The  initial  criteria  for  respondent  sample  was  a  maximum  of  two  tours  of  duty  (up  to  six  years)  after 
graduation  from  the  program.  A  stratified  random  sample  was  then  taken  from  the  remaining  the  96  ETMS 
officers  that  met  the  criteria  listed  above.  The  stratification  process  was  based  on  rank  of  officers 
(lieutenants  and  Lieutenant  Commanders)  to  ensure  a  equally  representative  sample  from  those  officers 
who  met  the  initial  criteria.    Once  stratified  the  sample  was  taken  using  the  random  numbers  table.  40 
questionnaires  were  mailed  out  to  ETMS  Subspecialists. 

The  respondents  for  the  supervisors  questionnaires  were  selected  using  the  list  of  ETMS  coded  billets  listed 
in  a  Navy  officer  billet  listing  publication.  As  only  15  billets  were  identified  for  ETMS  Subspecialists  only 
fifteen  questionnaires  were  sent  out,  one  to  the  supervisors  at  each  command. 

The  respondents  for  the  interviews  were  randomly  selected  from  the  remaining  ETMS  officer  population. 
Both  officers  were  working  in  XX37P  coded  billets.  Both  were  contacted  and  interviewed  by  phone  using 
the  questionnaire  as  a  format  for  the  closed  and  open  response  questions. 

Data  Collection 

Three  types  of  data  were  collected  for  a  triangulation  of  information.  The  first  type  was  the  questionnaires 
mentioned  above.  These  were  in  a  semi-structured  format  with  a  predominance  of  agreement  scale 
questions  followed  by  open  response  probes  at  the  end.  The  questions  asked  the  respondents  to  rate  their 
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acquisition  level  of  10  ESR's,  as  identified  as  the  Navy's  ETMS  program  objectives,  and  their  relevancy  to 
fleet  education  jobs.  The  open-response  portion  of  the  questionnaires  asked  the  sample  for  their  personal 
ETMS  history  information,  their  opinions  of  the  program,  and  their  recommendations  for  program 
improvement. 

The  second  collection  method  was  also  agreement  scale  questionnaires,  however  these  were  targeted  at 
fifteen  supervisors  that  oversee  ETMS  officers  in  fleet  education  jobs.  This  questionnaire  followed  the 
same  format  as  above,  only  supervisors  rated  their  subordinates  on  their  mastery  of  the  the  10  ESR's  as 
well  as  the  supervisors  rating  their  perceptions  of  skill's  relevancy  to  the  job.  It  also  ended  with  an  open 
response  section. 

Finally  two  interviews  were  conducted.    The  interview,  in  a  highly  structured  format,  paralleled  the 
questionnaires  with  the  emphasis  on  open  responses  to  all  questions.  The  first  part  of  the  interview  used 
the  same  format  of  the  agreement  scale  questions  used  above.  This  was  followed  by  the  same  open-ended 
questions  that  were  also  used  at  the  end  of  the  questionnaires. 


Instrument  Objectives 


The  following  questions  were  addressed  by  this  evaluation  study  via  the  two  questionnaires  and  the 
interviews: 

•  Is  the  knowledge  level  specified  by  the  ten  educational  skill  areas  acquired  through  the  ETMS 
program? 

•  Are  the  skills  attained  from  the  ETMS  program  relevant  and  applicable  to  fleet  education  jobs? 

•  What  did  Navy  ETMS  officers  like  and  dislike  about  the  program  and  what  would  they  do  to 
improve  it? 

Additional  information  collected  at  the  clients  request  include: 

•  University  attended  during  ETMS  program  participation. 

•  How  long  between  graduation  and  utilization  tour. 

•  What  is  more  career  enhancing:  department  head  or  utilization  tour. 

•  What  continuing  education  has  been  pursued. 


Data  Analysis 


The  data  collected  was  analyzed  by  two  methods.  The  closed  response  data  collected  in  the 
questionnaires  was  scored  and  rated  on  knowledge  level,  and  the  applicability  and  relevance  (both  graduate 
and  supervisor  ratings)  of  the  10  skill  areas  to  education  jobs.  Values  were  assigned  to  responses  on  a 
scale  of  1  to  5.  One  being  very  low,  five  being  very  high.  Number  and  percentage  of  responses  were 
assigned  to  ESR's  by  respondent  category.  Mean  scores  were  also  assigned  for  each  ESR  rating  by 


respondent  category.  The  open  responses  were  analyzed  with  the  those  from  the  interviews  using  the 
responses  themselves  to  create  categories  or  patterns  of  response.  Trends  were  identified  and  then  analyzed 
against  the  closed  responses  for  potential  commonality. 

Limitations  of  Study 

The  major  limitation  of  this  evaluation  is  it's  potential  lack  of  generalizability.  This  is  due  to  the  low 
number  of  usable  questionnaires  returned  by  the  sample.  Besides  the  normal  disinterest  most  people  have 
for  completing  and  returning  questionnaires  a  secondary  element  was  identified  as  the  cause  for  low 
usability  of  responses.  The  sample  was  selected  from  a  computer  print  out  titled  "XX37P  Coded  Officers" 
that  was  obtained  from  the  Graduate  Education  Placement  Department  of  the  Navy  Military  Personnel 
Command  at  Washington  D.C.  As  questionnaires  were  returned,  it  became  apparent  that  the  list  sent  from 
Washington  contained  not  only  XX37P  coded  officers,  but  XX37S  coded  officers  as  well.  The  "S"  code 
identifies  that  the  individual  received  their  subspecialty  rating  not  from  a  school  program,  but  through 
experience  alone.  Table  1  demonstrates  the  response  rates. 


Table  1 
Questionnaire  Response  Rate 


Sample 
Total 

N=55 

Responses 

N=27 

Percent 
Response 

ETMS  Graduates 

Lieutenant  Commanders 

20 

11 

55% 

Lieutenants 

20 

10 

50% 

Total 

40 

21 

55% 

(Usable) 

(15) 

(37%) 

ETMS  Officer  Supervisors 

15 

6 

40% 

Basically  despite  the  fact  of  a  55%  questionnaire  return  rate  from  the  ETMS  officers,  after  removing  the 
XX37S  officers  from  the  returned  data,  only  37%  of  those  returned  were  usable.  Due  to  time  restrictions  it 
was  not  possible  to  request  a  second  "purified"  list  of  P  coded  officers  and  re-select  the  sample. 


FINDINGS 


The  following  information  was  collected  and  divided  into  three  primary  sections  to  address  the  major 
evaluation  questions:  Is  the  knowledge  level  specified  by  the  ten  educational  skill  areas  acquired  through 
the  ETMS  program?;  Are  the  skills  and  knowledge  attained  from  the  ETMS  program  relevant  and  applicable 
to  fleet  education  jobs?;  What  did  Navy  ETMS  officers  like  and  dislike  about  the  program  and  what  would 
they  do  to  improve  it?  The  additional  demographic  information  collected  at  the  client  s  request  can  be  found 
in  Appendix  A. 

Is  the  knowledge  level  specified  by  the  ten  ESR's  acquired  through  the 
ETMS  program? 

Educational  skill  requirements  were  rated  by  the  sample  based  on  level  of  knowledge  attained  and  level  of 
relevance  to  fleet  jobs.  The  findings  from  the  data  were  further  divided  by  ETMS  officers  self  rated 
information,  and  ETMS  Officer  Supervisor  rated  information.  Finally  all  data  collected,  including 
interviews  were  compared  in  tables  under  each  heading. 

Budget  constraints  and  fleet  needs  dictate  that  the  Navy  educational  programs  not  only  identify  specific 
knowledge  required  for  a  field,  but  ensure  that  it  is  acquired  by  program  participants.  Table  2  displays  the 
rate  of  ESR  knowledge  acquired  as  identified  by  actual  ETMS  officers  based  on  number  and  percent  of 
responses  to  each  knowledge  acquisition  level.  The  major  portion  of  the  respondents  rated  their  knowledge 
acquisition  of  ESR's  as  moderate  to  high.  The  noticeable  exceptions  are  ESR's  2,3,  and  9.  All  three  of 
these  skill  requirements  (Naval  Education  and  Training  organization,  Planning  Programming  and 
Budgeting  System,  and  Contract  Administration)  are  very  uniquely  "Navy"  oriented  which  might  help 
explain  the  lack  of  this  type  of  education  in  civilian  university  programs.  This  table  also  illustrates  the 
central  tendency  in  scoring,  with  a  very  strong  inclination  for  a  "moderate"  rating  average. 


Table  2 

Level  of  ESR  Knowledge  Acquisition 

(Self-Rated) 

N=15 

Educational  Skill  Requirement  Level 

Very 

High      High 

Moderate 

Low 

Very 
Low 

No 
Opinion 

% 
(N) 

(N) 

% 
(N) 

% 
(N) 

% 
(N) 

% 
(N) 

1 .  Knowledge  of  basic  Management 
Principles: 

6.5% 
(1) 

47% 
(7) 

40% 
(6) 

6.5% 
(1) 

0% 

0% 

2.  Knowledge  of  Naval  Education  and 
Training  (NAVEDTRA)  organization  and 
operations: 

0% 

13% 
(2) 

27% 
(4) 

20% 

(3) 

40% 

(6) 

0% 

3.  Knowledge  of  Planning,  Programming 
and  Budgeting  System  (PPBS): 

6.5% 
(1) 

0% 

20% 

(3) 

40% 
(6) 

27% 
(4) 

6.5% 
(1) 

4.  Knowledge  of  the  principles  and 
applications  of  Instructional  Systems 
Development  (ISD): 

27% 
(4) 

13% 
(2) 

20% 

(3) 

20% 

(3) 

20% 

(3) 

0% 

5.  Knowledge  of  Personnel/Manpower 
management  policies  and  procedures: 

20% 
(3) 

27% 
(4) 

40% 

(6) 

6.5% 
(1) 

6.5% 
(1) 

0% 

6.  Knowledge  of  training/educational 
psychology: 

47% 
(7) 

33% 
(5) 

13% 
(2) 

6.5% 
(1) 

0% 

0% 

7.  Knowledge  of  the  Training  Research  and 
Development  process: 

0% 

33% 
(5) 

47% 
(7) 

13% 

(2) 

0% 

6.5% 
(1) 

8.  Knowledge  of  the  applications  of 
computer  technology  to  Education/Training 
management: 

13% 

(2) 

20% 

(3) 

40% 

(6) 

13% 
(2) 

13% 
(2) 

0% 

9.  Knowledge  on  Contract  Administration: 

0% 

6.5% 
(1) 

20% 

(3) 

40% 

(6) 

27% 
(4) 

6.5% 
(1) 

10.  Knowledge  on  existing  and  conceptual 
training  systems: 

13% 
(2) 

6.5% 
(1) 

40% 

(6) 

27% 
(4) 

0% 

13% 

(2) 

Table  3  quantifies  the  same  type  of  data  as  rated  by  ETMS  officer  supervisors. 


Table  3 

Level  of  ESR  Knowledge  Acquisition 

(Supervisor  Rated) 

N=6 


Very 
High 

High 

Moderate 

Low 

Very 
Low 

No 
Opinion 

Educational  Skill  Requirement  Level 

% 
(N) 

% 
(N) 

% 
(N) 

% 
(N) 

% 
(N) 

% 
(N) 

1 .  Knowledge  of  basic  Management 
Principles: 

67% 
(4) 

33% 
(2) 

0% 

0% 

0% 

0% 

2.  Knowledge  of  Naval  Education  and 
Training  (NAVEDTRA)  organization  and 
operations: 

50% 
(3) 

33% 
(2) 

16.5% 

(1) 

0% 

0% 

0% 

3.  Knowledge  of  Planning,  Programming 
and  Budgeting  System  (PPBS): 

16.5% 
(1) 

50% 

(3) 

16.5% 
(1) 

0% 

16.5% 
(1) 

0% 

4.  Knowledge  of  the  principles  and 
applications  of  Instructional  Systems 
Development  (ISD): 

67% 
(4) 

33% 
(2) 

0% 

0% 

0% 

0% 

5.  Knowledge  of  Personnel/Manpower 
management  policies  and  procedures: 

83% 
(5) 

16.5% 
(1) 

0% 

0% 

0% 

0% 

6.  Knowledge  of  training/educational 
psychology: 

83% 
(5) 

16.5% 
(1) 

0% 

0% 

0% 

0% 

7.  Knowledge  of  the  Training  Research  and 
Development  process: 

67% 
(4) 

16.5% 
(1) 

16.5% 
(1) 

0% 

0% 

0% 

8.  Knowledge  of  the  applications  of 
computer  technology  to  Education/Training 
management: 

16.5% 
(1) 

50% 
(3) 

16.5% 
(1) 

16.5% 
(1) 

0% 

0% 

9.  Knowledge  on  Contract  Administration: 

16.5% 
(1) 

16.5% 
(1) 

16.5% 
(1) 

16.5% 
(1) 

33% 
(2) 

0% 

10.  Knowledge  on  existing  and  conceptual 
training  systems: 

67% 
(4) 

16.5% 
(1) 

16.5% 
(1) 

0% 

0% 

0% 

This  table  shows  that  the  majority  of  supervisors  rate  their  ETMS  officers  as  having  a  moderate  to  very 
high  level  of  ESR  knowledge  acquisition.  Only  one  skill  area  was  rated  below  this,  skill  requirement 
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number  9,  which  was  rated  low  to  moderate.  As  stated  before,  Navy  Contract  Administration  can  be  very 
specific  in  terms  of  Navy  regulations  which  would  make  certain  aspects  of  it  difficult  to  be  offered  as  a 
regular  course  at  a  civilian  university.  It  is  also  apparent  that  supervisors  rate  the  knowledge  of  their  ETMS 
officers  more  highly  than  the  actual  officers  themselves.  Table  4  is  an  abbreviated  summary  of  the  number 
and  percentages  of  the  responses  for  all  of  the  respondents  as  a  comparison  of  rating  levels. 

Table  4 

Knowledge  Acquisition:    Abbreviated  Response  Summary 

ETMS  officers,  Interviewees,  and  Supervisors 


ESR  #  and  Respondents 

High  to  Very 
High 

%(N) 

Moderate 

%(N) 

Low  to  Very  Low 
%(N) 

No  Opinion 

%(N) 

ESR  #  1.    ETMS  officers 
Interviewees 
Supervisors 

53.5%  (8) 
100%  (2) 
100%  (6) 

40%  (6) 
0% 
0% 

6.5%  (1) 
0% 
0% 

0% 
0% 
0% 

ESR  #  2.    ETMS  officers 
Interviewees 
Supervisors 

13%  (2) 

0% 
83%  (5) 

27%  (4) 

50%  (1) 

16.5%  (1) 

60%  (9) 

50%  (1) 

0% 

0% 
0% 
0% 

ESR  ft  3.    ETMS  officers 
Interviewees 
Supervisors 

6.5%  (1) 
100%  (2) 
67%  (4) 

20%  (3) 

0% 
16.5%  (1) 

67%  (10) 

0% 
16.5%  (1) 

6.5%  (1) 
0% 
0% 

ESR  #  4.    ETMS  officers 
Interviewees 
Supervisors 

40%  (6) 
100%  (2) 
100%  (6) 

20%  (3) 
0% 
0% 

40%  (6) 

13%  (2) 

0% 

0% 
0% 

0% 

ESR  ft  5.    ETMS  officers 
Interviewees 
Supervisors 

47%  (7) 
100%  (2) 
100%  (6) 

40%  (6) 
0% 
0% 

13%  (2) 
0% 
0% 

0% 
0% 
0% 

ESR  #6.    ETMS  officers 
Interviewees 
Supervisors 

80%  (12) 
100%  (2) 
100%  (6) 

13%  (2) 
0% 
0% 

6.5%  (1) 
0% 
0% 

0% 
0% 
0% 

ESR  #7.    ETMS  officers 
Interviewees 
Supervisors 

33%  (5) 
50%  (1) 
83%  (5) 

47%  (7) 

50%  (1) 

16.5%  (1) 

13%  (2) 
0% 
0% 

6.5%  (1) 
0% 
0% 

ESR  8  8.     ETMS  officers 
Interviewees 
Supervisors 

33%  (5) 

0% 
67%  (4) 

40%  (6) 
100%  (2) 
16.5%  (1) 

27%  (4) 

0% 
16.5%  (1) 

0% 
0% 
0% 

ESR  #  9.    ETMS  officers 
Interviewees 
Supervisors 

6.5%  (1) 
50%  (1) 
33%  (2) 

20%  (3) 

50%  (1) 

16.5%  (1) 

67%  (10) 

0% 
67%  (3) 

6.5%  (1) 
0% 
0% 

ESR  #  10.  ETMS  officers 
Interviewees 
Supervisors 

20%  (3) 
50%  (1) 
83%  (5) 

40%  (6) 

50%  (1) 

16.5%  (1) 

27%  (4) 
0% 
0% 

13%  (2) 
0% 
0% 

Here  by  including  the  interviews,  the  wide  range  of  response  is  evident.  A  better  way  to  view  this 
information  is  using  the  mean  scores  assigned  for  all  three  sample  groups,  as  shown  in  Figure  1.  An 
interesting  feature  identified  here  is  that  the  interview  responses  are  much  closer  in  rating  value  to  the 
supervisors  than  that  of  the  rest  of  the  ETMS  officers  who  responded  to  the  questionnaires. 

Figure  1 


Level  of  Knowledge  of  Educational  Skill  Requirements 

Very  High     5- 


Supervisors  Questionnaires 
E53  ETMS  Questionnaires 
^  ETMS  Interviews 


Educational  Skill  Requirements 


This  figure  demonstrates  the  vast  variation  of  opinions  on  skill  requirement  acquisition.  Because  of  the 
limited  number  of  responses  on  which  it  is  based,  it  is  very  difficult  to  identify  any  commonality  in  the 
ratings.  Those  skill  areas  with  the  most  obvious  similarity  in  means  are  ESR  6  -  Education  and  Training 
psychology,  rated  high  to  very  high,  and  ESR  7  -  Training  Research  and  Development  Processes  rated 
moderate  to  high.  The  overall  mean  for  the  ESR  knowledge  level  is  3.6  or  moderate  to  high. 

Are  the  skills  and  knowledge  attained  from  the  ETMS  program  relevant 
and  applicable  to  fleet  education  jobs? 

The  second  major  area  of  information  collected  was  on  the  perceived  level  of  ESR  relevancy  to  fleet  jobs. 
The  first  part  of  this  discussion  of  findings  will  focus  in  on  ESR  relevancy  as  rated  on  the  ETMS  officers 
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questionnaires.  This  area  of  report  had  a  great  potential  for  variability  due  to  the  varying  jobs  and  degrees 
of  skills  required  in  each. 

Table  5 

Level  of  ESR  Relevancy  to  Fleet  Job 

(Self  Rated) 

N=15 


Verv 
High 

High 

Moderate 

Low 

Very 
Low 

No 
Opinion 

Educational  Skill  Requirements 

% 
(N) 

% 
(N) 

% 
(N) 

% 
(N) 

% 
(N) 

% 
(N) 

1 .  Relevance  of  basic  Management 
Principles  to  the  job: 

27% 
(4) 

47% 
(7) 

20% 

(3) 

6.5% 
(1) 

0% 

0% 

2.  Relevance  of  Naval  Education  and 
Training  (NAVEDTRA)  organization  and 
operations  to  the  job: 

20% 
(3) 

13% 

(2) 

27% 
(4) 

13% 
(2) 

33% 
(5) 

0% 

3.  Relevance  of  Planning,  Programming 
and  Budgeting  System  (PPBS)  to  the  job: 

0% 

27% 
(4) 

27% 
(4) 

20% 

(3) 

20% 

(3) 

6.5% 
(1) 

4.  Relevance  of  the  principles  and 
applications  of  Instructional  Systems 
Development  (ISD)  to  the  job: 

27% 
(4) 

13% 
(2) 

6.5% 
(1) 

47% 
(7) 

6.5% 
(1) 

0% 

5.  Relevance  of  Personnel/Manpower 
management  policies  and  procedures  to  the 
job: 

40% 
(6) 

40% 

(6) 

13% 
(2) 

6.5% 
(1) 

0% 

0% 

6.  Relevance  of  training/educational 
psychology  to  the  job: 

20% 

(3) 

47% 
(7) 

20% 

(3) 

0% 

13% 
(2) 

0% 

7.  Relevance  of  the  Training  Research  and 
Development  process  to  the  job: 

0% 

27% 
(4) 

27% 
(4) 

27% 
(4) 

13% 
(2) 

6.5% 
(1) 

8.  Relevance  of  the  applications  of 
computer  technology  to  Education/Training 
management  to  the  job: 

13% 
(2) 

47% 
(7) 

13% 
(2) 

13% 

(2) 

13% 

(2) 

0% 

9.  Relevance  on  Contract  Administration  to 
the  job: 

6.5% 
(1) 

27% 
(4) 

6.5% 
(1) 

20% 

(3) 

33% 
(5) 

6.5% 
(1) 

10.  Relevance  on  existing  and  conceptual 
training  systems  to  the  job: 

13% 
(2) 

6.5% 
(1) 

40% 

(6) 

13% 
(2) 

13% 
(2) 

13% 
(2) 

This  table  demonstrates  that  for  nine  out  of  ten  of  the  skill  areas,  more  than  half  of  the  respondents  rated 
them  very  highly  relevant.  The  exception  being  Contract  Administration  which  was  rated  by  53%  as  low  to 
very  low  in  job  relevance.  This  table  especially  interesting  with  it's  consistent  polarity  of  the  ETMS  officer 
responses.  This  could  be  due  to  the  fact  that  just  less  than  half  (as  identified  later  on  in  this  report)  of  the 
respondents  had  ever  even  worked  in  an  XX37P  (ETMS)  coded  billet  The  overall  mean  average  for  all  ten 

11 


skill  areas  was  3.12,  or  slightly  higher  than  a  moderate  level  of  job  relevancy.  Table  6  lists  the  responses 

on  relevancy  as  perceived  by  ETMS  Officer  supervisors. 

Table  6 

Level  of  ESR  Relevancy  to  Fleet  Job 

(Supervisor  Rated) 

N=6 


Very 

High 

High 

Moderate 

Low 

Very 
Low 

No 
Opinion 

Educational  Skill  Requirements 

% 

(N) 

% 
(N) 

% 
(N) 

% 
(N) 

% 
(N) 

% 
(N) 

1 .  Relevance  of  basic  Management 
Principles  to  the  job: 

83% 
(5) 

16.5% 
(1) 

0% 

0% 

0% 

0% 

2.  Relevance  of  Naval  Education  and 
Training  (NAVEDTRA)  organization  and 
operations  to  the  job: 

50% 
(3) 

16.5% 
(1) 

16.5% 
(1) 

0% 

16.5% 
(1) 

0% 

3.  Relevance  of  Planning,  Programming 
and  Budgeting  System  (PPBS)  to  the  job: 

16.5% 
(1) 

33% 
(2) 

16.5% 
(1) 

16.5% 
(1) 

16.5% 
(1) 

0% 

4.  Relevance  of  the  principles  and 
applications  of  Instructional  Systems 
Development  (ISD)  to  the  job: 

16.5% 
(1) 

33% 
(2) 

0% 

33% 
(2) 

16.5% 
(1) 

0% 

5.  Relevance  of  Personnel/Manpower 
management  policies  and  procedures  to  the 
job: 

50% 
(3) 

50% 

(3) 

0% 

0% 

0% 

0% 

6.  Relevance  of  training/educational 
psychology  to  the  job: 

67% 
(4) 

33% 
(2) 

0% 

0% 

0% 

0% 

7.  Relevance  of  the  Training  Research  and 
Development  process  to  the  job: 

0% 

0% 

50% 

(3) 

33% 
(2) 

16.5% 
(1) 

0% 

8.  Relevance  of  the  applications  of 
computer  technology  to  Education/Training 
management  to  the  job: 

0% 

16.5% 
(1) 

33% 
(2) 

16.5% 
(1) 

33% 
(2) 

0% 

9.  Relevance  on  Contract  Administration  to 
the  job: 

33% 
(2) 

16.5% 
(1) 

0% 

50% 
(3) 

0% 

0% 

10.  Relevance  on  existing  and  conceptual 
training  systems  to  the  job: 

16.5% 
(1) 

33% 
(2) 

16.5% 
(1) 

16.5% 
(1) 

16.5% 
(1) 

0% 

This  table  once  again  shows  that  Supervisors  (with  two  exceptions)  consistently  rate  ESR's  higher  than 
their  ETMS  officers.  The  majority  of  the  skill  areas  were  rated  at  a  moderate  to  very  high  level  of  relevance 
to  fleet  jobs.  The  overall  mean  score  for  all  ten  of  the  ESR's  was  3.5  or  moderate  to  high.  The  exception 
to  this  were  ESR  7  -  Training  Research  and  Development,  and  ESR  8  -  Computer  Technology.  Both  were 
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rated  as  low  in  job  relevance.  Table  7  once  again  summarizes  all  the  responses  from  the  sample  on  on  ESR 
relevancy  to  fleet  jobs. 

Table  7 

ESR  Relevancy  to  Job:   Abbreviated  Response  Summary 

ETMS  officers,  Interviewees,  and  Supervisors 


ESR  #  and  Respondents 

High  to  Very 
High 

%(N) 

Moderate 

%(N) 

Low  to  Very  Low 
%(N) 

No  Opinion 

%(N) 

ESR  ft  1. 

ETMS  officers 
Interviewees 
Supervisors 

73%  (11) 
100%  (2) 
100%  (6) 

20%  (3) 
0% 
0% 

6.5%  (1) 
0% 
0% 

0% 
0% 
0% 

ESR  ft  2. 

ETMS  officers 
Interviewees 
Supervisors 

33%  (5) 
50%  (1) 
67%  (4) 

27%  (4) 

0% 
16.5%  (1) 

47%  (7) 

50%  (1) 

16.5%  (1) 

0% 
0% 
0% 

ESR  ft  3. 

ETMS  officers 
Interviewees 
Supervisors 

27%  (4) 
100%  (2) 
50%  (3) 

27%  (4) 
0% 
0% 

40%  (6) 

0% 
50%  (3) 

6.5%  (1) 
0% 
0% 

ESR  ft  4. 

ETMS  officers 
Interviewees 
Supervisors 

40%  (6) 
50%  (1) 
50%  (3) 

6.5%  (1) 
0% 
0% 

53%  (8) 
50%  (1) 
50%  (3) 

0% 
0% 
0% 

ESR  ft  5. 

ETMS  officers 
Interviewees 
Supervisors 

80%  (12) 
100%  (2) 
100%  (6) 

13%  (2) 
0% 
0% 

6.5%  (1) 
0% 
0% 

0% 
0% 
0% 

ESR  #6. 

ETMS  officers 
Interviewees 
Supervisors 

67%  (10) 
100%  (2) 
100%  (6) 

20%  (3) 
0% 
0% 

13%  (2) 
0% 
0% 

0% 
0% 
0% 

ESR  #7. 

ETMS  officers 
Interviewees 
Supervisors 

27%  (4) 

50%  (1) 

0% 

27%  (4) 

0% 
50%  (3) 

40%  (6) 
50%  (1) 
50%  (3) 

6.5%  (1) 
0% 
0% 

ESR  it  8. 

ETMS  officers 
Interviewees 
Supervisors 

60%  (9) 

50%  (1) 

16.5%  (1) 

13%  (2) 

0% 
33%  (2) 

27%  (4) 
50%  (1) 
50%  (3) 

0% 
0% 
0% 

ESR  ft  9. 

ETMS  officers 
Interviewees 
Supervisors 

33%  (5) 
100%  (2) 
50%  (3) 

6.5%  (1) 
0% 
0% 

53%  (8) 

0% 
50%  (3) 

65%  (1) 
0% 
0% 

ESR  ft  10 

.  ETMS  officers 
Interviewees 
Supervisors 

20%  (3) 
50%  (1) 
50%  (3) 

40%  (6) 

0% 
163%  (1) 

27%  (4) 
50%  (1) 
33%  (2) 

13%  (2) 
0% 
0% 

This  table  points  out  that  most  of  the  respondents  (again  with  two  exceptions)  rated  ESR  to  job  relevancy  as 
moderate  to  very  high.  It  is  also  more  consistent  and  shows  more  commonality  in  opinion  than  the  earlier 
representation  on  knowledge  acquisition.  This  is  even  more  apparent  in  Figure  2,  which  is  a  bar  graph  of 
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the  ESR  means  for  each  set  of  respondents.  This  chart  graphically  demonstrates  the  particular  similarity  in 
ratings  of  most  of  the  skill  areas  except  ESR's  3  and  9. 

Figure  2 
Level  of  Relevance  of  Educational  Skill  Requirements 

Very  High       5- 


High 


Moderate 


Low 


Very  Low 


S  u  pe  r vi  so  rs  Q  uesti  o  n  nai  rea 
f£3  ETMS Questionnaires 
fflh  ETMS  Interviews 


3  4  5  6  7  8 

Educational  Skill  Requirements 


The  commonality  shared  in  the  majority  of  ratings  of  the  ESR's  job  relevance  seem  to  point  to  it  as  being 
more  universally  measured  by  both  ETMS  officers  and  their  Supervisors.  The  overall  mean  for  the  10  skill 
areas  is  a  3.5,  or  a  moderate  to  high  relevancy  factor.  The  two  low  to  moderate  rated  areas  were  Training 
Research  and  Development  (ESR  7),  and  Computer  Technology  Applications  (ESR  8). 

Program  Positive/Negative  Comments  and  Recommended 
Improvements 

The  final  section  of  the  findings  and  the  questionnaire/interviews  related  to  the  perceived  "good  and  bad" 
aspects  of  the  ETMS  program.  The  following  tables  summarize  the  most  frequently  cited  positive  and 
negative  open-responses  at  the  end  of  the  questionnaires  and  interviews.  For  a  complete  summary  of  all 
responses,  see  Appendix  C. 
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Table  8 
Positive  ETMS  Program  Open  Responses 


(N) 

1 .  2  Interesting  subject  variety  and  selection 

2 .  2  Good  adult  learning  and  educational  psychology  courses 

3 .  2  Outstanding  ISD  instruction 

4 .  2  Business  courses  were  the  most  valuable 

5 .  2  Graduates  have  an  extensive  background  in  Educational  Training  and  Management 


The  second  and  third  responses  are  useful  in  that  they  support  data  collected  in  the  agreement  scale  portion 
of  the  questionnaires  and  interviews.  Response  number  two  directly  reflects  the  overall  mean  rating  of 
knowledge  acquisition  of  high  to  very  high  in  ESR  6-  Training  and  Educational  psychology.  Response 
number  three  is  interesting  in  that  it  coincides  with  the  overall  mean  rating  of  high  in  knowledge  acquisition 
for  ESR  4.  Response  number  five  lightly  contradicts  the  overall  mean  rating  of  moderate  to  high  for  the 
knowledge  acquisition  of  ESR  1,  Management  Principles. 

Table  1 1  lists  the  most  frequently  cited  negative  responses  about  the  ETMS  program. 

Table  9 
Negative  ETMS  Program  Open  Responses 


(N) 


2.  4 

3.  4 

4.  3 

5.  3 

6.  3 


Many  course  options  aren't  relevant  to  program  (ie.  School  Law,  Computer  Programming, 

Client/consultant  courses) 

University  ETMS  program  advisor  is  not  helpful 

Courses  geared  strictly  to  civilian  programs/needs  (primarily  teachers  and  principals) 

No  courses  on  Contract  Administration  -  need  for  job 

Management  courses  redundant  for  Naval  Officers 

Frequently  courses  most  relevant  to  ESR's  no  offered  (during  program  completion  time) 


The  comments  made  in  response  one,  that  courses  are  frequently  not  applicable  to  the  program  (computer 
programming  is  one  of  the  courses  cited),  supports  the  overall  low  to  moderate  relevancy  rating  of  ESR  8  - 
Computer  Technology.  On  the  other  hand  it  does  not  support  the  low  to  moderate  rating  of  knowledge 
level  for  the  same  skill  area-  Response  four  about  Contract  Administration  supports  the  overall  very  low  to 
low  rating  for  knowledge  level  of  ESR  9,  and  becomes  an  important  statement  when  compared  to  an  overall 
rating  of  moderate  to  high  in  job  relevancy.  Response  number  five  on  management  courses,  is  balanced  by 
an  overall  rating  for  ESR  1  (Management  Principles)  of  Moderate  to  High  in  knowledge  level  but  a  high  to 
very  high  level  of  relevancy.  The  final  response  is  especially  interesting  in  that  it  supports  the  low  to 
moderate  rated  knowledge  level  ESR's  (2-  NAVEDTRA  Organization,  3-  Planning  Programming  and 
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Budgeting  System  and  9-  Contract  Administration).  The  last  response  validates  the  need  for  courses  that 
meed  the  curricula  standards  for  ESR's  2,3,  and  9. 

This  last  table  summarizes  the  most  frequently  mentioned  comments  on  changes  to  improve  the  program 
that  the  respondents  said  that  they  would  like  to  see.  As  above  a  complete  summary  of  all  responses  can  be 
found  in  Appendix  C. 

I  Table  10 

Recommended  Change  Responses  to  Improve  ETMS  Program 


(N) 

Add  courses  on: 

1.  4  Contract  Administration 

2.  3  Program  evaluation 

3.  2  How  Navy  training  courses  curriculum  are  designed  and  developed 

4.  2  ISD  (specially  test  development  and  objective  writing 

5.  2  Navy  Training  ESR  course 

6.  2  Educational  Technology  (development  and  training  aids) 


The  most  frequently  cited  responses  all  involve  course  additions.  While  responses  one  and  three  are 
supported  by  previously  mentioned  low  to  moderate  rated  knowledge  level  ESR's,  responses  four  and  six 
are  contradicted  by  the  low  to  moderate  job  relevancy  of  their  related  ESR's. 
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CONCLUSIONS  &  RECOMMEND  A  TIONS 


1.  Based  on  the  small  size  of  the  useable 
response  from  the  sample,  this  evaluation 
may  not  be  generalizable  to  the  entire 
ETMS  program  and  it's  community  of 
subspecialists. 


A  secondary  evaluation  should  be  conducted 
targeting  only  and  all  of  the  XX37P  coded 
officers. 


2.  More  than  half  of  the  responses  were 
received  from  ETMS  officers  who  have 
never  been  in  a  utilization  tour. 


Naval  Military  Personnel  Command  should  be 
alerted  to  this  and  make  adjustments  in  the 
assignment  procedure  to  ensure  that 
subspecialists  are  given  a  utilization  tour  within 
the  first  two  tours  after  graduation  as  specified 
by  OPNAV. 


The  follow  up  evaluation  should  be  of 
sufficient  size  that  only  the  responses  of 
XX37P  coded  officers  who  have  been  or  are  in 
a  utilization  tour  are  used  to  determine  the 
relevance  of  their  education  to  fleet  ETMS  jobs. 


3.  Three  Educational  Skill  Requirements 
(Contract  Administration,  Personnel 
Planning  and  Budgeting  System,  and 
NAVEDTRA  Organization)  are  not 
sufficiently  addressed  by  most  civilian 
universities. 


Arrangements  should  be  made  with  the  civilian 
institutions  to  ensure  that  their  ETMS  program 
curricula  meets  all  of  the  required  objectives. 

An  optional  recommendation  is  that  an  ESR 
course  be  taught  at  all  host  civilian  institutions 
(or  at  a  local  base)  by  Navy  instructors.  An 
example  of  just  such  a  course  has  been  created 
by  and  XX37P  coded  officer  and  will  be  taught 
this  fall  at  Old  Dominion  University.  For  more 
information  See  Appendix  E. 
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LITERATURE  REVIEW 


The  concept  of  relevancy  of  education  to  job  needs  is  of  great  importance  to  both  universities  and 
employers.  The  majority  of  literature  that  supports  this  statement  falls  into  two  broad  categories;  graduate's 
evaluation  of  the  match  between  curricula  and  required  job  skills,  and  employers  evaluation  of  graduates 
job  skills,  performance,  and  knowledge. 

Wheelock  and  Zekeri  (1988)  conducted  a  study  that  attempted  to  determine  which  competencies  former 
students  found  most  essential  to  their  careers.  They  hypothesized  that  the  more  that  is  known  about 
competencies  needed  in  careers  and  are  taken  into  account  in  curriculum  development  the  more  competitive 
graduates  could  be.  According  to  Cedergrin  ( 1985)  continuing  follow-up  has  great  value  to  any  college, 
yet  he  feels  that  follow-up  studies  of  graduates  to  determine  their  perceptions  of  the  strengths  and 
weaknesses  of  curricula  are  infrequently  undertaken.  In  his  book,  Konrad  (1975)  expressed  the  opinion 
that  data  that  are  generated  from  follow-up  studies  should  be  utilized  as  soon  as  they  are  available  and  then 
stored  in  a  management  information  system  until  further  analysis  can  be  undertaken.  In  an  article  by  Hoyt 
(1987),  results  from  a  1987  Gallup  poll  are  cited  that  indicate  only  about  half  of  the  employed  college 
graduates  said  that  their  skills  and  abilities  were  being  used  "very  well"  in  their  current  jobs.  Houghton's 
studies  (1971)  of  the  business  program  at  Southern  Oregon  College  found  that  the  weakest  aspects  of  the 
program  in  the  opinion  of  the  graduates  were  that  the  courses  were  too  theoretical  and  the  curriculum  was 
poor.  In  another  recent  study  (Gerbert  1988)  surveys  were  used  to  identify  the  topics  that  new  dentists 
believed  to  be  under-emphasized  in  the  dental  school  curriculum  as  well  as  those  that  are  most  important  to 
dental  practice. 

Carnoy  ( 1979)  researched  the  role  of  education  in  meeting  employment  objectives  in  the  hopes  of 
identifying  policy  oriented  planning  procedures  to  ensure  employer  needs  are  met  Also  hoping  that  the 
results  of  study  would  help  improve  educational  programs  to  better  meet  employers'  needs,  Travis  and 
Jones  (1976)  had  supervisors  evaluate  graduates  in  the  area  of  job  performance.  In  a  another  study  to 
determine  how  adequately  college  prepared  1978  graduates  for  employment  in  some  specific  job  skill  and 
areas  of  knowledge,  Campbell  and  Dogoloff  ( 1979)  found  that  employers  rated  94%  of  graduates 
performance  of  job  skills  as  "adequate  or  more  than  adequate."  Additionally,  the  survey  results  indicated 
that  while  supervisors  rated  the  overall  job  performance  of  graduates  as  adequate  or  more  than  adequate, 
they  cited  the  need  for  improvement  in  technical  and  problem  solving  skills. 

These  studies  demonstrate  the  ongoing  need  by  both  educational  institutions  and  employers  for  continuing 
research  that  evaluates  the  balance  between  employers  needs  and  the  curriculum  provided  for  students. 
Such  evaluations  will  help  create  a  dynamic  and  systematic  relationship  between  educational  institutions  and 
the  work  place. 
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Appendix  A.  -  Client  Requested  Demographics 
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This  next  section  of  findings  will  report  the  information  collected  specifically  at  the  clients  request. 

Current  Billet 

The  first  question  to  be  addressed  is  the  utilization  of  ETMS  officers  in  specific  XX37  coded  billets. 
Figure  3  is  a  pie  chart  representation  of  the  respondents  (ETMS  officers  only)  cited  tour  utilization  history. 
It  reveals  several  interesting  trends.  The  foremost  being  that  nearly  half  (41  %)  of  the  respondents  have  not 
had  a  utilization  tour  to  date.  This  is  especially  alarming  in  that  it  identifies  the  underutilization  of  a  readily 
available  and  costly  asset:  Navy  officers  with  masters  degrees  in  education  that  were  frequently  fully 
funded  by  the  government. 

Figure  3 

ETMS  Officer  XX37P  Billet  Utilization 


Current  Tour  41%  (7) 
Previous  Tour  20%  (3) 
Never  41%  (7) 


Of  the  17  ETMS  Officer  responses  (including  the  interviewees)  six  were  placed  in  utilization  tours 
immediately  after  graduation,  four  on  their  second  tour  after  graduation.  As  noted  above  the  remaining 
seven  have  never  had  a  utilization  tour.  Personal  preference  may  be  a  factor  in  this,  as  officers  frequently 
choose  billets  outside  of  their  subspecialty  for  various  reasons. 

Universities  Attended 

The  following  table  identifies  the  respondents  by  the  universities  they  attended  during  their  participation  in 
the  the  ETMS  program.  Only  five  of  the  possible  eight  universities  are  represented  in  the  sample,  but  each 
is  relatively  equal  in  representation. 


A-2 


Stanford 

%(N) 


20%  (3) 


Table  11 

Breakdown  of  ETMS  Graduate  Respondents 

by   University 

N=17* 


George 
Washington 

%(N) 


27%  (4) 


West  Florida 

%(N) 


27%  (4) 


Old  Dominion 

%(N) 


27%  (4) 


San  Diego 
State 

%(N) 


13%  (2) 


Continuing  Education 

Of  the  17  ETMS  officers  (including  interviewees)  only  2  had,  or  are  pursuing  continuing  education.  Of 
those  two,  one  is  taking  courses  in  Education  and  Human  Resources  Development,  the  other  has  completed 
a  doctorate  in  Education  Administration. 


Career  Enhancement 

This  question  is  very  pertinent  as  the  Navy  continues  to  shift  back  and  forth  from  a  strictly  leadership  track 
to  one  more  based  on  a  subspecialty  career  emphasis.  It  is  interesting  to  note  that  the  majority  of  both  the 
ETMS  officers  and  their  supervisors  cited  the  Department  Head  tour  as  more  career  enhancing.  The 
comments  made  by  the  three  respondents  that  answered  out  of  category,  were  particularly  helpful  in 
identifying  what  is  perceived  as  "the  best  of  both  worlds",  the  optimum  answer:  a  dual  Department 
Head/Utilization  tour.  Unfortunately  this  type  of  billet  is  considerably  rare  in  ETMS  assignments. 

Table  12 

ETMS  Officer  and  Supervisor 

Career  Concepts 

N=23* 


Most  Career  Enhancing 
after  Graduation? 

Subspecialty  Utilization 
Tour 

%  (N) 

Department  Head 
Tour 

%  (N) 

ETMS  officers 

27%  (4) 

60%  (9) 

ETMS  Officer  Interviewees 

0% 

100%  (2) 

ETMS  Officer  Supervisors 

17%  (1) 

66%  (4) 

Total 

35%   (5) 

52%   (15) 
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*  Two  of  the  ETMS  officers  and  one  Supervisor  wrote  in  that  the  "ideal  career  enhancing  tour"  after 
graduation  would  be  a  Department  Head/Utilization  tour. 
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Appendix  B.  -  Questionnaires 
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ETMS  Officer  Questionnaire 


Try  to  remember  back,  and  based  solely  on  your  ETMS  degree  education  for  the  XX37P  subspecialty, 
please  rate  the  the  following  Education  Skill  Requirements  by  the  level  of  knowledge  you  acquired  and  it's 
applicability  and  relevance  to  your  current  job.  Please  read  each  skill  area  and  circle  the  level  that  most 
closely  matches  yours.  There  are  six  possible  responses:  Very  High,  High,  Moderate,  Low,  Very  Low, 
and  No  Opinion. 

At  the  end  there  are  eight  general  information  questions,  followed  by  space  for  your  recommendations  on 
improving  the  ETMS  program.  Feel  free  to  add  any  additional  comments  that  you  feel  would  be  pertinent. 
Thank  you  again  for  your  time. 


1 .  Level  of  knowledge  of  basic  Management  Principles: 

•  fundamental  management  theories 

•  management  system  basic  elements 

•  resource  management  system 

2.  Level  of  relevance  of  basic  Management  Principles  to 
your  job. 

3.  Level  of  knowledge  of  Naval  Education  and  Training 
(NAVEDTRA)  organization  and  operations: 

•  organizational  structure 

•  training  program  implementation  procedures 

•  training  program  maintenance 

•  training  requirement  validation 

•  education  and  training  roles  of  Navy  commands 

4.  Level  of  relevance  of  NAVEDTRA  organization  and 
operations  knowledge  to  your  job. 

5.  Level  of  knowledge  of  Planning,  Programming  and 
Budgeting  System  (PPBS): 

•  translating  validation  into  acquisition 

6.  Level  of  relevance  of  PPBS  to  your  job. 

7.  Level  of  knowledge  of  the  principles  and  applications  of 
Instructional  Systems  Development  (ISD): 

•  Tenets  of  instructional  technology  and  ISD 

•  ISD  advantages 

•  instructor  roles  with  ISD 

8.  Level  of  relevance  of  ISD  to  your  job. 


Very  High  High  Moderate  Low  Very  Low  No  Opinion 

Very  High  High  Moderate  Low  Very  Low  No  Opinion 

Very  High  High  Moderate  Low  Very  Low  No  Opinion 

Very  High  High  Moderate  Low  Very  Low  No  Opinion 

Very  High  High  Moderate  Low  Very  Low  No  Opinion 

Very  High  High  Moderate  Low  Very  Low  No  Opinion 

Very  High  High  Moderate  Low  Very  Low  No  Opinion 

Very  High  High  Moderate  Low  Very  Low  No  Opinion 

Very  High  High  Moderate  Low  Very  Low  No  Opinion 

Very  High  High  Moderate  Low  Very  Low  No  Opinion 

Very  High  High  Moderate  Low  Very  Low  No  Opinion 

Very  High  High  Moderate  Low  Very  Low  No  Opinion 

Very  High  High  Moderate  Low  Very  Low  No  Opinion 

Very  High  High  Moderate  Low  Very  Low  No  Opinion 

Very  High  High  Moderate  Low  Very  Low  No  Opinion 

Very  High  High  Moderate  Low  Very  Low  No  Opinion 

Very  High  High  Moderate  Low  Very  Low  No  Opinion 

Very  High  High  Moderate  Low  Very  Low  No  Opinion 

Very  High  High  Moderate  Low  Very  Low  No  Opinion 

Very  High  High  Moderate  Low  Very  Low  No  Opinion 
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9.  Level  of  knowledge  of  Personnel/Manpower  management 
policies  and  procedures: 

•  theories  of  manpower  management 

•  tenets  of  personnel  administration 

10.  Level  of  relevance  of  basic  Personnel/Manpower 
management  policies  and  procedures  to  your  job. 

1 1 .  Level  of  knowledge  of  training/educational  psychology: 

•  learning  processes 

•  learning  theory 

12.  Level  of  relevance  of  Training/Educational  Psychology 
to  your  job. 

13.  Level  of  knowledge  of  the  Training  Research  and 
Development  process: 

•  steps  for  the  research  and  development  process 

•  research  and  development  initiation  and  guidance 

14.  Level  of  relevance  of  the  Training  Research  and 
Development  process  to  your  job. 

15.  Level  of  knowledge  of  the  applications  of  computer 
technology  to  Education/Training  management: 

•  ADP  systems 

•  computer  theory  and  system  analysis 

•  management  of  computer  applications 

•  computer  capability  /limitations 

•  computer  assisted  instruction 

16.  Level  of  relevance  of  computer  technology  knowledge  to 
your  job. 

17.  Level  of  knowledge  on  Contract  Administration: 

•  procurement  management 

•  procurement  regulations 

•  product  evaluation 

1 8.  Level  of  relevance  of  Contract  Administration  to  your 
job. 


Very  High     High     Moderate      Low      Very  Low     No  Opinion 


Very  High  High  Moderate 

Very  High  High  Moderate 

Very  High  High  Moderate 

Very  High  High  Moderate 

Very  High  High  Moderate 

Very  High  High  Moderate 

Very  High  High  Moderate 


Low  Very  Low  No  Opinion 

Low  Very  Low  No  Opinion 

Low  Very  Low  No  Opinion 

Low  Very  Low  No  Opinion 

Low  Very  Low  No  Opinion 

Low  Very  Low  No  Opinion 

Low  Very  Low  No  Opinion 


Very  High     High     Moderate      Low      Very  Low    No  Opinion 


Very  High  High  Moderate 
Very  High  High  Moderate 
Very  High     High     Moderate 


Low  Very  Low  No  Opinion 
Low  Very  Low  No  Opinion 
Low      Very  Low    No  Opinion 


Very  High     High     Moderate      Low      Very  Low    No  Opinion 


Very  High 
Very  High 
Very  High 
Very  High 
Very  High 
Very  High 


High  Moderate 

High  Moderate 

High  Moderate 

High  Moderate 

High  Moderate 

High  Moderate 


Very  High  High  Moderate 

Very  High  High  Moderate 

Very  High  High  Moderate 

Very  High  High  Moderate 

Very  High  High  Moderate 


Low  Very  Low  No  Opinion 

Low  Very  Low  No  Opinion 

Low  Very  Low  No  Opinion 

Low  Very  Low  No  Opinion 

Low  Very  Low  No  Opinion 

Low  Very  Low  No  Opinion 

Low  Very  Low  No  Opinion 

Low  Very  Low  No  Opinion 

Low  Very  Low  No  Opinion 

Low  Very  Low  No  Opinion 

Low  Very  Low  No  Opinion 
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19.  Level  of  knowledge  on  existing  and  conceptual  training       Very  High     High     Moderate      Low      Very  Low    No  Opinion 
systems: 

•  non  ISD  training  theories:  assets/limitations  Very  High     High     Moderate      Low      Very  Low    No  Opinion 

20.  Level  of  relevance  of  knowledge  on  existing  and  Very  High     High     Moderate      Low      Very  Low    No  Opinion 
conceptual  training  systems  to  your  job. 

21.  Are  you  currently  in  an  XX37P  coded  billet? 

•  If  no,  when  were  you  last  in  an  XX37P  coded  billet? 

22.  What  university  did  you  attend  during  your  participation  in  the  ETMS  program? 

23.  How  long  was  it  after  graduation  before  your  XX37P  utilization  tour? 

24.  Have  you  pursued  continuing  education  since  graduation? 

•  If  yes,  what  courses  and  in  what  field? 


25.  What  would  have  been  more  career  enhancing  for  you  after  graduation;  a  department  head  or  utilization  tour? 

26.  Was  there  anything  that  you  especially  liked  about  the  curricula  offered  at  your  university  for  ETMS  program? 


27.  Was  there  anything  that  you  especially  disliked  about  the  curricula  offered  at  your  university  for  ETMS  program? 
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28.  If  you  could,  what  changes/improvements  would  you  make  to  the  ETMS  program,  or  the  university's  curriculum  to  make 
it  more  job  relevant? 


Please  feel  free  to  make  any  additional  comments. 


Thank  you  again  for  your  help. 
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ETMS  Supervisor  Questionnaire 


Based  on  your  knowledge  of  and  experience  with  the  XX37P  officer  who  works  for  you,  please  rate- 
him/her  on  the  following  Education  Skill  Requirements  by  the  level  of  knowledge  displayed  to  you.  Also 
please  rate  the  level  of  applicability  and  relevance  of  these  skills  to  his/her  job.  Please  read  each  skill  area 
and  circle  the  level  that  most  closely  matches.  There  are  six  possible  responses:  Very  High,  High, 
Moderate,  Low,  Very  Low,  and  No  Opinion. 

At  the  end  there  are  six  general  information  questions,  followed  by  space  for  any  recommendations  you 
may  have  for  improving  the  ETMS  program/curricula.  Feel  free  to  add  any  additional  comments  that  you 
feel  would  be  pertinent.  Thank  you  again  for  your  time. 


1.  Level  of  knowledge  of  basic  Management  Principles: 

•  fundamental  management  theories 

•  management  system  basic  elements 

•  resource  management  system 

2.  Level  of  relevance  of  basic  Management  Principles  to  the 
job. 

3.  Level  of  knowledge  of  Naval  Education  and  Training 
(NAVEDTRA)  organization  and  operations: 

•  organizational  structure 

•  training  program  implementation  procedures 

•  training  program  maintenance 

•  training  requirement  validation 

•  education  and  training  roles  of  Navy  commands 

4.  Level  of  relevance  of  NAVEDTRA  organization  and 
operations  knowledge  to  the  job. 

5.  Level  of  knowledge  of  Planning,  Programming  and 
Budgeting  System  (PPBS): 

•  translating  validation  into  acquisition 

6.  Level  of  relevance  of  PPBS  to  the  job. 

7.  Level  of  knowledge  of  the  principles  and  applications  of 
Instructional  Systems  Development  (ISD): 

•  Tenets  of  instructional  technology  and  ISD 

•  ISD  advantages 

•  instructor  roles  with  ISD 

8.  Level  of  relevance  of  ISD  to  the  job. 


Very  High  High  Moderate 

Very  High  High  Moderate 

Very  High  High  Moderate 

Very  High  High  Moderate 

Very  High  High  Moderate 


Low  Very  Low  No  Opinion 

Low  Very  Low  No  Opinion 

Low  Very  Low  No  Opinion 

Low  Very  Low  No  Opinion 

Low  Very  Low  No  Opinion 


Very  High     High     Moderate      Low  Very  Low  No  Opinion 

Low  Very  Low  No  Opinion 

Low  Very  Low  No  Opinion 

Low  Very  Low  No  Opinion 

Low  Very  Low  No  Opinion 

Low  Very  Low  No  Opinion 

Low  Very  Low  No  Opinion 

Very  High     High     Moderate      Low  Very  Low  No  Opinion 


Very  High 

High 

Moderate 

Very  High 

High 

Moderate 

Very  High 

High 

Moderate 

Very  High 

High 

Moderate 

Very  High 

High 

Moderate 

Very  High 

High 

Moderate 

Very  High  High  Moderate 

Very  High  High  Moderate 

Very  High  High  Moderate 

Very  High  High  Moderate 

Very  High  High  Moderate 

Very  High  High  Moderate 

Very  High  High  Moderate 


Low  Very  Low  No  Opinion 

Low  Very  Low  No  Opinion 

Low  Very  Low  No  Opinion 

Low  Very  Low  No  Opinion 

Low  Very  Low  No  Opinion 

Low  Very  Low  No  Opinion 

Low  Very  Low  No  Opinion 
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9.  Level  of  knowledge  of  Personnel/Manpower  management 
policies  and  procedures: 

•  theories  of  manpower  management 

•  tenets  of  personnel  administration 

10.  Level  of  relevance  of  basic  Personnel/Manpower 
management  policies  and  procedures  to  the  job. 

1 1 .  Level  of  knowledge  of  training/educational  psychology: 

•  learning  processes 

•  learning  theory 

12.  Level  of  relevance  of  Training/Educational  Psychology 
to  the  job. 

13.  Level  of  knowledge  of  the  Training  Research  and 
Development  process: 

•  steps  for  the  research  and  development  process 

•  research  and  development  initiation  and  guidance 

14.  Level  of  relevance  of  the  Training  Research  and 
Development  process  to  the  job. 

1 5 .  Level  of  knowledge  of  the  applications  of  computer 
technology  to  Education/Training  management: 

•  ADP  systems 

•  computer  theory  and  system  analysis 

•  management  of  computer  applications 
■  computer  capability/limitations 

•  computer  assisted  instruction 

16.  Level  of  relevance  of  computer  technology  knowledge  to 
the  job. 

17.  Level  of  knowledge  on  Contract  Administration: 

•  procurement  management 

•  procurement  regulations 

•  product  evaluation 

18.  Level  of  relevance  of  Contract  Administration  to  the  job. 


Very  High     High     Moderate      Low      Very  Low    No  Opinion 


Very  High 
Very  High 
Very  High 

Very  High 
Very  High 
Very  High 
Very  High 


High  Moderate 

High  Moderate 

High  Moderate 

High  Moderate 

High  Moderate 

High  Moderate 

High  Moderate 


Low  Very  Low  No  Opinion 

Low  Very  Low  No  Opinion 

Low  Very  Low  No  Opinion 

Low  Very  Low  No  Opinion 

Low  Very  Low  No  Opinion 

Low  Very  Low  No  Opinion 

Low  Very  Low  No  Opinion 


Very  High     High     Moderate      Low     Very  Low    No  Opinion 


Very  High 
Very  High 
Very  High 


High  Moderate 
High  Moderate 
High     Moderate 


Low  Very  Low  No  Opinion 
Low  Very  Low  No  Opinion 
Low     Very  Low    No  Opinion 


Very  High     High     Moderate      Low      Very  Low    No  Opinion 


Very  High 
Very  High 
Very  High 
Very  High 
Very  High 
Very  High 

Very  High 
Very  High 
Very  High 
Very  High 
Very  High 


High  Moderate 

High  Moderate 

High  Moderate 

High  Moderate 

High  Moderate 

High  Moderate 

High  Moderate 

High  Moderate 

High  Moderate 

High  Moderate 

High  Moderate 


Low  Very  Low  No  Opinion 

Low  Very  Low  No  Opinion 

Low  Very  Low  No  Opinion 

Low  Very  Low  No  Opinion 

Low  Very  Low  No  Opinion 

Low  Very  Low  No  Opinion 

Low  Very  Low  No  Opinion 

Low  Very  Low  No  Opinion 

Low  Very  Low  No  Opinion 

Low  Very  Low  No  Opinion 

Low  Very  Low  No  Opinion 
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19.  Level  of  knowledge  on  existing  and  conceptual  training       Very  High     High     Moderate      Low      Very  Low    No  Opinion 
systems: 

•  non  ISD  training  theories:  assets/limitations  Very  High     High     Moderate      Low      Very  Low     No  Opinion 

20.  Level  of  relevance  of  knowledge  on  existing  and  Very  High     High     Moderate      Low      Very  Low    No  Opinion 
conceptual  training  systems  to  the  job. 

21.  Do  you  know  what  university  your  XX37P  officer  attended  during  his/her  participation  in  the  ETMS  program? 

22.  Do  you  know  if  your  XX37P  officer  is/has  pursued  continuing  education  since  graduation? 

•  If  yes,  do  you  know  what  courses  and  in  what  field? 

23.  What  do  you  feel  is  more  career  enhancing  for  an  XX37P  officer  for  after  graduation;  a  department  head  or  utilization  tour? 

24.  Is  there  anything  that  positively  stands  out  about  your  XX37P  officer's  knowledge  and  ability? 


25.  Is  there  anything  that  negatively  stands  out  about  your  XX37P  officer's  knowledge  and  ability?? 
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26.  If  you  could,  what  changes/improvements  would  you  make  to  the  ETMS  program,  or  curriculum  to  make  it  more  job 
relevant? 


Please  feel  free  to  make  any  additional  comments. 


Thank  you  again  for  your  help. 


B-9 


Appendix  C.  ■    Questionnaire  and  Interview  Open  Response  Summary 
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N=23 

Positive  ETMS  Program  Responses 

Interesting  subject  variety  and  selection 

Good  adult  learning  and  educational  psychology  courses 

Outstanding  ISD  instruction 

Business  courses  were  the  most  valuable 

Graduates  have  an  extensive  background  in  Educational  Training  and  Management 

PPBS  course  offered  was  relevant  and  useful 

Practicum  for  Principals  was  the  best  course 

It  made  me  computer  literate  (finally) 

Outstanding  instructors 

ODU  program  is  more  adult  leaming/HRD  thus  more  relevant  than  other  programs 

Economics  of  Education  course  was  great 

Teacher  Evaluation  course  was  very  relevant  for  Navy  instructor  evaluations 

Good  educational  administration,  personnel  and  organizational  change  courses 

Statistics  course  was  very  useful  to  my  current  job. 

Excellent  cognitive  learning  theory 


Negative  ETMS  Program  Responses 

University  ETMS  program  advisor  is  not  helpful 

Courses  geared  strictly  to  civilian  programs/needs  (primarily  teachers  and  principals) 

Many  course  options  in  the  program  are  not  applicable  (ie.  School  Law,  Computer 

Programming,  Client/consultant  courses 

No  courses  on  Contract  Administration  -  need  for  job 

Management  courses  redundant  for  Naval  Officers 

Frequently  courses  most  relevant  to  ESR's  no  offered  (during  program  completion  time) 

Most  of  what  we  learned  is  not  relevant  or  used  on  the  job 

XX37P  officers  not  fully  utilized  in  fleet  jobs 

Insufficient  time  to  complete  the  program 

Civilian  instructors  have  no  experience  with  Navy  Education  an  Training 

ETMS  curriculum  is  disorganized 

No  mentors  available 

Business  courses  very  hard  to  enroll  in 
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#  Recommended  Change  Responses  to  Improve  ETMS  Program 

Add  courses  on: 
IV  Contract  Administration 

III  Program  evaluation 

II  How  Navy  training  courses  curriculum  are  designed  and  developed 

II  ISD  (specially  test  development  and  objective  writing 

II  Navy  Training  ESR  course 

II  Educational  Technology  (development  and  training  aids) 

3  or  4  elective  courses  for  area  of  concentration  (ED  or  HRD) 

Training  research  and  development 

Front  end  analysis 
II  Allow  more  time  for  program  completion 


General  Respondent  Recommendations  for  XX37's 

Create  a  specific  XX37  career  progression 

Ensure  first  tour  is  a  utilization  tour 

Ensure  First  tour  is  a  "schools  house"  tour 

CNET  should  track  XX37's  to  ensure  maximum  utilization  tours 
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Appendix  D.  -  Cover  Letters 
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9  April  1991 


Dear  Sir, 

I  am  a  graduate  student  in  the  Education  and  Training  Management  Subspecialty  (ETMS)  program 
at  San  Diego  State  University.  Currently  I  am  working  on  my  masters  project,  an  evaluation  of  the  ETMS 
program,  for  Dr.  J.D.  Smith  at  the  office  of  the  Chief  of  Naval  Education  and  Training  (CNET).  The  focus 
of  this  evaluation  is  to  determine  the  level  to  which  various  civilian  universities'  curricula  help  ETMS 
subspecialists  (XX37P  coded  officers)  acquire  knowledge  from  the  ten  education  skill  areas  defined  by 
CNET,  and  the  relevance  and  applicability  of  this  knowledge  to  an  XX37P  officers  job  in  the  fleet.  Your 
input  is  vital  to  this  evaluation. 

Enclosed  is  a  short  questionnaire,  and  a  return  addressed  envelope.  It  should  take  approximately  10 
to  15  minutes  to  complete.  If  you  do  not  directly  oversee  the  XX37P  officer  at  you  command,  please  ask 
the  responsible  department  head  to  complete  and  return  it  as  soon  as  possible.  If  there  are  more  than  one 
XX37P  officer  at  you  command,  if  you  wish,  please  make  copies  of  this  questionnaire  to  accommodate 
them.  If  you  would  like  a  copy  of  the  finished  evaluation  summary,  just  write  a  note  at  the  end  of  the 
questionnaire. 

This  evaluation  will  not  be  valid  without  your  input.  Thank  you  in  advance  for  you  assistance. 
With  your  participation,  this  evaluation  will  help  to  improve  the  ETMS  program  for  the  fleet  and  future 
graduates. 


Very  respectfully, 


Aron  L.  Gladney  ^X 


LT  USN 
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9  April  1991 


Dear  ETMS  Subspecialist, 

I  am  an  ETMS  graduate  student  currently  working  on  my  masters  project  at  San  Diego  State 
University.  The  project  is  an  evaluation  of  the  ETMS  program,  for  Dr.  J.D.  Smith  at  CNET.  The  focus  of 
the  evaluation  is  to  determine  the  level  to  which  the  various  universities'  curricula  helped  you  acquire  the 
knowledge  from  the  ten  education  skill  areas,  and  the  relevance  and  applicability  of  this  knowledge  to  your 
job  as  an  XX37P.  Your  input  is  vital  to  this  evaluation. 

Enclosed  is  a  short  questionnaire,  and  a  return  addressed  envelope.  It  should  take  you 
approximately  10  to  15  minutes  to  complete  it  If  you  are  not  currently  in  a  XX37P  coded  billet,  please 
answer  the  questions  relating  them  to  your  last  utilization  tour.  Please  return  it  as  soon  as  possible.  If  you 
would  like  a  copy  of  the  finished  evaluation  summary,  just  write  a  note  at  the  end  of  the  questionnaire. 

This  evaluation  will  not  be  valid  without  your  input.  Thank  you  in  advance  for  you  assistance  and 
expediency.  With  your  participation,  this  evaluation  will  help  to  improve  the  ETMS  program  for  future 
graduates. 


Very  respectfully, 


Aron  L.  Gladney 
LT  USN 
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Appendix  E.  -  ESR  Course  Addition  Information 
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FALL  SEMESTER  1991 
ETMS  STUDENTS 


ELS  697   Topics:   Perspectives  of  Navy  Training 

Index  Number:   58  3  08 

Wednesday,  4:20  -  6:50,  Batten  Arts  &  Letters  Building,  Room  331 

This  course  is  designed  to  incorporate  many  of  the  education  and 
training  objectives  in  the  Education,  Training,  and  Management 
Subspecialty  (ETMS)  curriculum  offered  at  ODU  into  a  concise  13 
week,  three  hour  course.  A  seminar  approach  will  be  used  with  much 
of  the  material  presented  by  guest  lecturers,  and  tours  of  local 
training  facilities  and  commands. 

General  Objectives; 

1.  Develop  an  understanding  of  the  various  innovative  appli- 
tions  to  Navy  training  and  appropriate  uses  of  these 
approaches. 

2.  Understand  the  Navy  requirements  for  training  in  the  21st 
century  and  new  approaches  to  meet  those  needs. 

3.  Become  familiar  with  the  Navy's  ETMS  billet  availabilities 
and  selected  career  assignments. 

4.  Consider  potential  applications  of  the  ETMS  training 
received  through  the  university  assignments. 

5.  Be  aware  of  selected  local  facilities  and  each  command's 
approach  towards  fulfilling  its  mission  and  supporting  Navy 
training. 

All  ETMS  students  should  plan  take  this  course  as  part  of  their  3  6 
hour  program. 
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Gladney 

Evaluation  of  the 
U.S.  Navy  Education  and 
Training  Management 
Subspecialty  program. 


